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Korat: autonnated testing based on Java predicates 
Chandrasekhar BoyapatI, Sarfraz Khurshid, Darko Marinov 

July 2002 ACM SIGSOFT Software Engineering Notes , Proceedings of the 2002 ACM 
SIGSOFT International symposium on Software testing and analysis ISSTA 

'02, Volume 27 Issue 4 
Publisher: ACM Press 

Full text available: ^pdf(171.43 KB) Additional Information: fijii citation , abstract , references , citings 

This paper presents Korat, a novel frannework for autonnated testing of Java progranns. 
Given a formal specification for a nnethod, Korat uses the method precondition to 
automatically generate all (nonisomorphic) test cases up to a given small size. Korat then 
executes the method on each test case, and uses the method postcondition as a test 
oracle to check the correctness of each output.To generate test cases for a method, Korat 
constructs a Java predicate (i.e., a method that returns a boolean) fr ... 



Unit testing frameworks 

Robert E. Noonan, Richard H. ProsI 

February 2002 ACM SIGCSE Bulletin , Proceedings of the 33rd SIGCSE technical 

symposium on Computer science education SIGCSE '02, volume 34 issue i 
Publisher: ACM Press 

Full text available: ^ pdf(4Q6.45 KB) Additional Information: full citation , abstract , references , citings 

In this paper we explore franneworks for performing unit testing in Java. The vehicle for 
this exploration is a student-written, skeleton program developed for the Computer 
Graphics course. Our analysis of this one experiment leads us to speculate what benefits 
in program development and design might accrue by requiring students to unit test their 
own programs. 



Object oriented framework development 
Marcus Eduardo Marklewicz, Carlos J. P. de Lucena 
July 2001 Crossroads, Volume 7 issue 4 

Publisher: ACM Press 
Full text available: ^ html(47.39 
KB) 
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Session 7B: agent analysis and design: Extreme programming of multi-aaent 

systems 
Holger Knublauch 

July 2002 Proceedings of the first international joint conference on Autonomous 
agents and multiagent systems: part 2 

Publisher: ACM Press 

Full text available: pdf(776.87 KB) Additional Information: full citation , abstract , references , index terms 

The complexity of communication scenarios between agents make nnulti-agent systems 
difficult to build. Most of the existing Agent-Oriented Software Engineering methodologies 
face this complexity by guiding the developers through a rather waterfall-based process 
with a series of intermediate modeling artifacts. While these methodologies lead to 
executable prototypes relatively late and are expensive when requirements change, we 
explore a rather evolutionary approach with explicit support for chang ... 

Keywords: agent-oriented software engineering 



5 The joint ACM SIGPLAN student research competition & OOPSLA poster session: Q 
Dawn: must J2EE-Webapplications be untestable? 
Christian Dedek, Andreas Spall, Sabine Winkler 

November 2002 Companion of the 17tli annual ACM SIGPLAN conference on Object- 
oriented programming, systems, languages, and applications 

Publisher: ACM Press 

Full text available: " gpdf^ 88.67 KB) Additional Information: full citation, abstract , references 

We present a cookbook for the development of J2EE-Web applications. This cookbook 
aims at the minimization of quality costs. It consists of procedures and guidelines in 
combination with a toolset. The classical software lifecycle presents test as a single phase 
in the development process. The cookbook presents testing as the motor of every other 
activity in development. 

Keywords: J2EE, Web engineering, agile processes, open source tools, test-methodology 




Full papers: Explicit programming I 
Avl Bryant, Andrew Catton, Kris De Voider, Gail C. Murphy 

April 2002 Proceedings of the 1st international conference on Aspect-oriented 
software development 

Publisher: ACM Press 

Full text available: ■g| ndf(856.41 KB^ Additional Information: abstract, references , dflngs. index 

Many design concepts can be expressed only indirectly in source code. When this occurs, 
a single concept at design results in a verbose amount of code that is scattered across the 
system structure. In this paper, we present explicit programming, an approach that 
enables a developer to introduce new vocabulary into the source to capture a design 
concept explicitly. An introduced vocabulary item modularizes the implementation details 
associated with a design concept, reducing the scattering of cod ... 

CartaBlanca— a pure-Java, component-based systems simulation tool for coupled 

non-linear physics on unstructured grids 

W. B. VanderHeyden, E. D. Dendy, N. T. Padlal-Collins 

June 2001 Proceedings of the 2001 joint ACM-ISCOPE conference on Java Grande 
Publisher: ACM Press 
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Full text available: '^ pdf(924.14 KB) Additional Information: full citation , abstract , references , citings , index 

terms 

This paper describes a component-based non-linear physical system simulation 
prototyping package written entirely in Java using object-oriented design to provide 
scientists and engineers a '"developer-friendly" software environment for large-scale 
computational method and physical model development. The software design centers on 
the Jacoblan-Free Newton-Krylov solution method surrounding a finite-volume treatment 
of conservation equations. This enables a clean component-based imp! ... 

Keywords: Jacobian, Java object oriented, Krylov, Newton, components, parallel, 
physics, solver, threads 



8 Is ISSTA research relevant to industrial users? panel ■ ISSTA 2002: empowering the Q 

developer to be a tester tool 

Vincent Encontre 

July 2002 ACM SIGSOFT Software Engineering Notes , Proceedings of the 2002 ACM 
SIGSOFT international symposium on Software testing and analysis ISSTA 
'02, Volume 27 Issue 4 
Publisher: ACM Press 

Full text available: ^pdf(1 44.54 KB^ Additional Information: full citation , abstract 

In this paper - scoped for the panel discussion at ISSTA 2002 - we are discussing some 
techniques to ease the adoption of testing techniques by the developers, by extending the 
debugging activity. We also briefly discuss a longer term vision where the same paradigm 
applies but at model level, when coding will be achieved using visual notations such as 
UML 

Keywords: JUnit, UML, debugging, runtime analysis, testing 



9 Technical correspondence: Embedding built-in tests in hot spots of an object-oriented Q 
framework 

Taewoong Jeon, Hyon Woo Seung, Sungyoung Lee 
August 2002 ACM SIGPLAN Notices, volume 37 issue 8 
Publisher: ACM Press 

Full text available: ^ pdf(793.62 KB) Additional Information: full citation , abstract , references 

This paper describes a scheme of encapsulating test support code as built-in test (BIT) 
components and embedding them into the hot spots of an object-oriented framework so 
that defects caused by the modification and extension of the framework can be detected 
effectively and efficiently through testing. The test components embedded Into a 
framework in this way increase the testability of the framework by making it easy to 
control and observe the process of framework testing. The proposed technlqu ... 

Keywords: built-in test (BIT), hook classes, object-oriented framework, testability 



10 DrJava: a lightweight pedagogic environment for Java Q 
Eric Allen, Robert Cartwright, Brian Stoler 

February 2002 ACM SIGCSE Bulletin , Proceedings of the 33rd SIGCSE technical 

symposium on Computer science education SIGCSE '02, volume 34 issue i 
Publisher: ACM Press 

Full text available: ^ pdf(451.91 KB) Additional Information: full citation, abstract , references, citings 

DrJava is a pedagogic programming environment for Java that enables students to focus 
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on designing programs, rather than learning how to use the environment. The 
environment provides a simple interface based on a "read-eval-print loop" that enables a 
programmer to develop, test, and debug Java programs in an interactive, incremental 
fashion. This paper gives an overview of DrJava including Its pedagogic rationale, 
functionality, and implementation. 

11 Session: Making RUP agile 
Michael Hirsch 

November 2002 OOPSLA 2002 Practitioners Reports 
Publisher: ACM Press 

Full text available: ^ pdf(926.79 KB^ Additional Information: full citation , abstract , reference^, index terms 

The Unified Development Process (USDP) and especially Its Implementation by Rational 
Software Corporation, the Rational Unified Process (RUP), Is a comprehensive process 
covering almost all aspects of software development projects. However, due to the great 
level of detail provided by RUP, many professionals do not consider RUP practical for 
small, fast paced projects. This paper reports the experiences with RUP on two small 
projects with teams of 3 to 5 developers. RUP proved to be adaptable to ... 

12 Session: Agile techniques to avoid firefighting at a start-up 
Joseph A. Blotner 

November 2002 OOPSLA 2002 Practitioners Reports 
Publisher: ACM Press 

Full text available: ISjl pdf (187.60 KB) Additional Information: full citation , abstract . Index terms 



This paper documents the creation and evolution of the (modified) agile methodology 
Implemented at Sabrix, Inc., a start-up enterprise-class software company. The company 
was delivering alpha quality software to early adopter customers on a weekly basis, using 
urgency as the primary driver for product development. New features and bug fixes were 
completed in a manner similar to a soccer game between two teams of five-year-olds, 
with everyone on the team chasing down the latest emergency. This pa ... 

Keywords: SDLC, agile methodology, development process, start-up company 



''^ Testing nnonadic code with QuickCheck Q 
Koen Claessen, John Hughes 

October 2002 Proceedings of tlie 2002 ACM SIGPLAN workshop on Haskell Haskell '02 

Publisher: ACM Press 

Full text available: ^ pdf(136.14 KB) Additional Information: full citation , abstract , references, citings 

QuickCheck Is a previously published random testing tool for Haskell programs. In this 
paper we show how to use It for testing monadic code, and in particular Imperative code 
written using the ST monad, QuickCheck tests a program against a specification: we show 
that QuickCheck's specification language is sufficiently powerful to represent common 
forms of specifications: algebraic, model-based (both functional and relational), and pre- 
/post-conditional. Moreover, all these forms of specif ... 

14 PL! workshops: Testing monadic code with QuickCheck Q 
Koen Claessen, John Hughes 

December 2002 ACM SIGPLAN Notices, volume 37 issue 12 
Publisher: ACM Press 

Full text available: ^ Ddf(206.81 KB) Additional Information: full citation, abstract, references 

QuickCheck is a previously published randonn testing tool for Haskell programs. In this 
paper we show how to use It for testing monadic code, and in particular Imperative code 
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written using the sr monad. Quicl<Check tests a program against a specification: we sliow 
that QuickCheck's specification ianguage is sufficiently powerful to represent common 
forms of specifications: algebraic, model-based (both functional and relational), and pre- 
/post-conditlonal. Moreover, all these forms of specif ... 

'IS Software and document engineering: The relevance of software documentation, tools Q 
^ and technologies: a survey 
^ Andrew Forward, Timothy C. Lethbridge 

November 2002 Proceedings of the 2002 ACM symposium on Document engineering 

Publisher: ACM Press 

Additional Information: full citation , abstract , references , citings, index 



Full text available: ' 

^ terms 

This paper highlights the results of a survey of software professionals. One of the goals of 
this survey was to uncover the perceived relevance (or lacl< thereof) of software 
documentation, and the tools and technologies used to maintain, verify and validate such 
documents. The survey results highlight the preferences for and aversions against 
software documentation tools. Participants agree that documentation tools should seek to 
better extract knowledge from core resources. These resources inci ... 

Keywords: documentation relevance, documentation survey, documentation 
technologies, program comprehension, software documentation, software engineering, 
software maintenance 



16 Columns: Surfing the net for software engineering notes 

❖ Mark Doernhoefer 
September 2002 ACM SIGSOFT Software Engineering Notes, volume 27 issue 5 

Publisher: ACM Press 

Full text available: 1p| pdf(1.67 MB) Additional Infomiation: full citation 



Short papers: Aspect-oriented programming with model checking 
Naoyasu Ubayashi, Tetsuo Tamai 

April 2002 Proceedings of tlie 1st international conference on Aspect-oriented 

software development 
Publisher: ACM Press 

Full text available: g Ddf(606.77 KB^ Additional Information: full citation , abstract, reference. cjtiDgs. iodex 
^ terms 

Aspect-oriented programming (AOP) is a programming paradigm such that crosscutting 
concerns including synchronization policies, resource sharing and performance 
optimizations over objects are modularized as aspects that are separated from objects. A 
compiler, called a weaver, weaves aspects and objects together into a program. In AOP, 
however, it is not easy to verify the correctness of a woven program because crucial 
behaviors are strongly influenced by aspect descriptions. In order to deal wit ... 

Keywords: checking frameworks, model checking, validation 
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1 Validation and verification: Unit testing in multi-agent systems using mock agents and 
^ aspects 

Roberta Coelho, Uira Kulesza, Arndt von Staa, Carlos Lucena 

May 2006 Proceedings of the 2006 international woricshop on Software engineering 
for large-scale multi-agent systems SELMAS '06 

Publisher: ACM Press 

Full text available: ^ pdf(169.18 KB) Additional Information: full citation , abstract , references , index terms 

In this paper, we present a unit testing approach for MASs based on the use of Mock 
Agents. Each Mock Agent is responsible for testing a single role of an agent under 
successful and exceptional scenarios. Aspect-oriented techniques are used, in our testing 
approach, to monitor and control the execution of asynchronous test cases. We present 
an implementation of our approach on top of JADE platform, and show how we extended 
JUnit test framework in order to execute JADE test cases. 



Keywords: aspect oriented programming, dependability, mock objects, unit testing 



2 Unit testing: test early, test often 
Michael Olan 

December 2003 Journal of Computing Sciences in Colleges, volume 19 issue 2 
Publisher: Consortium for Computing Sciences in Colleges 

Full text available: ^pdfd 54.89 KB) Additional Infonmation: full citation , abstract , references , index terms 

Testing Is a critical part of good software development, but often gets only minimal 
coverage in introductory programming courses. Unit testing and selected aspects of test- 
driven development can be used to improve learning and encourage emphasis on quality 
and correctness. Tools like JUnit significantly simplify the generation of test cases. An 
additional benefit for Instructors is that these tools can also be used to automate project 
grading. 
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Parameterized unit tests extend the current Industry practice of using closed unit tests 
defined as parameterless methods. Traditional closed unit tests are re-obtained by 
instantiating the parameterized unit tests. We have developed the prototype tool Unit 
Meister, which uses symbolic execution and constraint solving to automatically compute a 
minimal set of inputs that exercise a parameterized unit test given certain coverage 
criteria. In addition, the parameterized unit tests can be us ... 

Keywords: algebraic data types, automatic test input generation, constraint solving, 
symbolic execution, unit testing 
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In this paper we explore frameworks for performing unit testing in Java. The vehicle for 
this exploration Is a student-written, skeleton program developed for the Computer 
Graphics course. Our analysis of this one experiment leads us to speculate what benefits 
in program development and design might accrue by requiring students to unit test their 
own programs. 

Research papers: test & analysis I: An intensional approach to the specification of 
test cases for database applications 
David Willmor, Suzanne M. Embury 
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When testing database applications, in addition to creating in-memory fixtures it is also 
necessary to create an initial database state that is appropriate for each test case. 
Current approaches either require exact database states to be specified in advance, or 
else generate a single initial state (under guidance from the user) that is intended to be 
suitable for execution of all test cases. The first method allows large test suites to be 
executed In batch, but requires considerable programmer ... 
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Unit testing Is a scalable and effective way to uncover software faults. In the JNuke 
project, automated regression tests connblned with coverage measurement ensured high 
code quality throughout the project. By using a custom testing environment, functionality 
was extended beyond what is commonly available by unit test frameworks. Low-overhead 
memory leak detection was implemented through wrapping. Automated support for log 
files made it possible to track the internal state of objects, which Is o ... 

Keywords: system testing, test framework, unit testing 
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Unit testing is one of the core practices in the Extreme Programming lightweight software 
development method, and it is usually carried out with the help of software frameworks 
that ease the construction of test cases as an Integral part of programming tasks. This 
work describes our first results in studying the Integration of automated unit testing 
practices in conventional 'introduction to programming' laboratories. Since the work used 
a classical procedural language in the course's assignment ... 
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^ terms 

Parameterized unit tests extend the current industry practice of using closed unit tests 
defined as parameterless methods. Parameterized unit tests separate two concerns: 1) 
They specify the external behavior of the Involved methods for all test arguments. 2) Test 
cases can be re-obtained as traditional closed unit tests by instantiating the 
parameterized unit tests. Symbolic execution and constraint solving can be used to 
automatically choose a minimal set of inputs that exercise a parameterize ... 

Keywords: algebraic data types, automatic test input generation, constraint solving, 
symbolic execution, unit testing 
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The Business Process Execution Language (BPEL) is emerging as the new standard in Web 
service composition. As more and more workflows are modelled using BPEL, unit-testing 
these compositions becomes Increasingly Important. However, little research has been 
done in this area and no frameworks comparable to the xUnit family are available. In this 
paper, we propose a layer-based approach to creating frameworks for repeatable, white- 
box BPEL unit testing, which we use for the development of a new te ... 
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Test case prioritization techniques schedule test cases for regression testing in an order 
that Increases their ability to meet some performance goal. One performance goal, rate of 
fault detection, measures how quickly faults are detected within the testing process. In 
previous work we provided a metric, APFD, for measuring rate of fault detection, and 
techniques for prioritizing test cases to improve APFD, and reported the results of 
experiments using those techniques. This met ... 

Keywords: fault severity, rate of fault detection, regression testing, test case 
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Automated tests can play a key role in ensuring system quality in software development. 
However, significant problems arise in automating tests of stochastic algorithms. 
Normally, developers write tests that simply check whether the actual result is equal to 
the expected result (perhaps within some tolerance). But for stochastic algorithms, 
restricting ourselves in this way severely limits the kinds of tests we can write: either to 
trivial tests, or to fragile and hard-tounderstand tests that re ... 

Keywords: hypothesis testing, software engineering, software testing, stochastic 
algorithms, unit tests 
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analysis ISSTA'06 

Publisher: ACM Press 
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This paper extends basic software-testing theory to software components and adds 
explicit state to the theory. The resulting theory is simple enough to abstractly model the 
construction of systems from their parts ('units'). It provides an unconventional insight 
into the relationship between testing units and testing systems. Experiments exploring the 
theory support the following conclusions: 

• Units should be Independent, more like what are called "components" than 
subroutines or object-o ... 

Keywords: persistent state, testing theory, unit/system testing 



15 Session 1 : Tool support for randomized unit testing 

James H. Andrews, Susmita Haldar, Yong Lei, Felix Chun Hang Li 

July 2006 Proceedings of the 1st international workshop on Random testing RT '06 
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There are several problem areas that must be addressed when applying randomization to 
unit testing. As yet no general, fully automated solution that works for all units has been 
proposed. We therefore have developed RUTE-J, a Java package intended to help 
programmers do randomized unit testing in Java. In this paper, we describe RUTE-J and 
illustrate how it supports the development of per-unit solutions for the problems of 
randomized unit testing. We report on an experiment in which we applied R ... 

Keywords: randomized testing, unit testing 
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While the usage of unit-testing frameworks such as JUnit has greatly increased over the 
last several years, it is not immediately apparent to students and instructors how to best 
use tools like JUnit and how to integrate testing across a computer science curriculum. We 
have worked over the last four semesters to infuse testing and JUnit across our 
curriculum, building from having students use JUnit to having them write their own test 
cases to building larger integration and use case testing syst ... 

Keywords: JUnit, testing, unit testing, unit testing frameworks 
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A prinnary goal of generative programming and model-driven development is to raise the 
level of abstraction at which designers and developers Interact with the software systems 
they are building. During initial development, the benefits of abstraction are clear. 
However, during testing and maintenance, increased distance from the implementation 
can be a disadvantage. We view test cases and test harnesses as an essential bridge 
between the high-level specifications and the implementation. As such, ... 
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Amit Paradkar 

November 1996 Proceedings of the 1996 conference of the Centre for Advanced 
Studies on Collaborative research 
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This paper describes an algorithm for testing polynnorphic Interactions between classes in 
object-oriented (O-O) software. We propose an Augmented Object Relationship Diagram 
(AORD), which is an extension of a previously reported ORD representation, to facilitate 
the algorithm. AORD is a graph representation of the class structure augmented with 
additional Information about the methods defined in each class. This additional 
information consists of call-graph of all the method ... 

19 Automated testing of virtual reality application interfaces B 
^ Allen Bierbaum, Patrick Hartling, Carolina Cruz-Neira 

^ May 2003 Proceedings of the workshop on Virtual environments 2003 EGVE '03 
Publisher: ACM Press 

Full text available: gpdf(601 .80 KB) Additional Information: full citation , abstract , references, index terms 

We describe a technique for supporting testing of the interaction aspect of virtual reality 
(VR) applications. Testing is a fundamental development practice that forms the basis of 
many software engineering methodologies. It is used to ensure the correct behavior of 
applications. Currently, there is no common pattern for automated testing of VR 
application interaction. We review current software engineering practices used In testing 
and explore how they may be applied to the specific realm of VR ... 

Keywords: VR Juggler, extreme programming, unit testing 
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At-speed testing of GHz processors using external testers may not be technically and 
economically feasible. Hence, there is an emerging need for low-cost, high-quality self- 
test methodologies, which can be used by processors to test themselves at-speed. 
Currently, Built-in Self-Test (BIST) Is the primary self-test methodology available and is 
widely used for testing embedded memory cores. In this paper, we report our experiences 
in applying a commercial BIST methodology to two processor cor ... 
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